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ABSTRACT 

This research study developed a value-added instructional improvement analysis model. 
North Carolina state testing results were used in regression and residual analyses of 
student achievement. This analysis allowed for identification of the “most effective ” and 
“least effective ” biology teachers in Wake County Public Schools (WCPSS). The study 
found that the “most effective ” biology teachers were focused on the delivery of biology 
instruction; resisted distractions from their classroom efforts; maximized student use of 
class time; studied and planned with other teachers using the North Carolina Biology 
Standard Course of Study; focused all student time on the Standard Course of Study 
goals; carefully planned teacher controlled student activities; and used data to guide 
their instructional practice. 


INTRODUCTION 

Many variables contribute to student success, but most experts now agree that the teacher 
is a key factor. Several studies of student gains from one year to another on standardized 
tests find the student’s assigned teacher to be the most influential factor (Rivkin, 
Hanushek, & Kain , 2001; Sanders & Horn, 1994; Sanders & Rivers, 1996; Wright, Horn, 
& Sanders, 1997). The importance of teachers in facilitating student success is 
recognized by the Elementary and Secondary Education Act of 1965 (ESEA), as 
amended by the No Child Left Behind Act of 2001 (NCLB). Under NCLB, every state 
must develop and implement a plan that ensures that all students will be taught by a 
“highly qualified teacher” (HQT; sec. 2101). Margaret Spellings , U. S. Secretary of 
Education, stated in a letter dated October 21, 2005, “There is also evidence that states 
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are improving the quality of their teaching forces.” She wrote that the U.S. Department 
of Education is committed to the goal of every child being taught by an HQT by the end 
of 2005-06, and then outlined the actions that will be taken to support states in reaching 
this goal (Spellings, 2005). 

The importance of teachers is also recognized in the National Science Education 
Standards. Chapter four is devoted to the standards for the professional development of 
teachers. Professional Development Standard C gives a list of musts for professional 
development activities. Among the list are “Provide opportunities for teachers to receive 
feedback about their teaching and to understand, analyze, and apply that feedback to 
improve their practice” (National Research Council, 1996). 

In North Carolina, all public high school teachers are required to follow a Standard 
Course of Study that specifies the goals and objectives of each course. For 10 of these 
courses, students are also required to take an End-of-Course (EOC) examination based on 
these goals and objectives. This requirement has been in place for several years. Thus, a 
database has been created that allows calculation of average scores for classes taught by 
any teacher over time. As part of a statewide school improvement project that predates 
the NCLB, the state of North Carolina provides monetary incentives to teachers in 
schools in which students attain specified levels of academic growth as demonstrated 
primarily on EOC tests. 

This research project used the state database in district regression and residual analyses of 
student achievement. This analysis allowed for identification of the “most effective” 
biology teachers and “least effective” biology teachers in WCPSS. Observation of 
classroom practice of “most effective” teachers produced meaningful feedback for all 
teachers to use in improving their classroom practice. 

PURPOSE OF THE RESEARCH 

The main goal of this research study was to develop a value-added instructional 
improvement analysis model. Unlike most current valued-added models, teacher 
performance evaluation was not a goal of this study (Braun, 2005; Olson, 2005; Olson, 
2004; Sanders, 1998; Tucker & Stronge, 2005). The value-added model incorporated 
collaboration between the WCPSS’ Evaluation and Research Department and the 
Curriculum and Instruction Department. Classroom instructional practices of “most 
effective” and “least effective” biology teachers, identified by the Evaluation and 
Research Department on the basis of aggregated student residuals, were observed, 
contrasted, and shared with school principals and teachers to facilitate discussion and 
improved practice. The observations were led by the staff of the Curriculum and 
Instruction Department. 
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BACKGROUND 

WHY START WITH BIOLOGY? 

Students must take a variety of courses to graduate from high school in North Carolina, 
but not all of these courses have an associated EOC exam. Of the required courses for 
which an EOC exam is used (Algebra I, Biology, Civics and Economics, English I, and 
U.S. History), Biology had the largest achievement gap between White and Black 
students. In 2004-05, 87.4% of WCPSS White students scored at proficiency but only 
47.2% of WCPSS Black students scored at proficiency. 

Biology is one of the 10 high school courses tested in the North Carolina ABC 
accountability program, the initiative designed to improve school outcomes. The 
program rewards schools and educators when students’ academic achievement meets or 
exceeds growth standards established for each student group or class. Performance on 
EOC exams, percent of students graduating prepared for college, and reduction of 
students dropping out are all considered in calculating ABCs results (North Carolina 
Department of Public Instruction [NCDPI], 2005). In WCPSS, Biology had the second 
largest number of enrolled students of the eight EOC courses tested in 2004-05 (Civics 
and Economics and U.S. History were not tested from 2003 to 2005). The district 
achieved expected, but not high, growth in Biology. Disaggregation of performance by 
race showed that Black students did not make expected growth, but the high growth of 
White students was sufficient to overcome this deficit. Four WCPSS high schools did 
not make expected growth in Biology and 1 1 other schools made expected, but not high, 
growth. The fact that the district’s board of education has embraced its own performance 
goal — Goal 2008 — which calls for high growth by all student subgroups, intensifies the 
pressure on high schools to improve outcomes in biology for all students. 

DEFINING EFFECTIVENESS 

In the State 

In North Carolina, EOC exams are administered in 10 high school subjects including 
biology. Each exam is a standardized multiple-choice test written with much input from 
teachers across the state. Teachers participate in test development in a variety of ways, 
from writing curriculum to writing and reviewing test items. Each student’s test is 
assigned a scale score based on the number of items correct and the difficulty of items. 
The scale scores are then converted to one of four levels of performance. Levels III and 
IV are associated with adequate or higher proficiency in the course content, and are 
considered proficient (NCDPI, 2004). 

Teachers receive rosters of students scale scores, level scores, and a 100-point scale 
score. An average scale score is also reported at the bottom of each roster. The 
percentages of students passing each EOC and all the EOCs in a school are reported 
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publicly. Teachers judge their own success using these percentages. The scores can also 
be disaggregated into many subgroups (i.e. students with disabilities, free or reduced- 
price lunch students, academically gifted students, etc). 

If 90% of the EOC scores at a school are in Level III or IV, the school is eligible to be 
labeled a “School of Excellence” by the state’s accountability program. However, 
because the program has two standards of achievement (the absolute percentage of tests 
at or above grade level and the attainment of “expected” growth), the aggregated scores 
of students tested must also meet or exceed the school’s growth prediction across all EOC 
subjects. The predicted growth is calculated on the basis of the history of previous tests 
taken by students. For the biology EOC, for example, the test scores used in the 
prediction model are 8th-grade reading and mathematics scores. If the school makes 
expected growth, all teachers receive a monetary bonus, regardless of the percentage of 
students testing at or above grade level. Note that the growth targets are based on the 
growth of students in the school; thus setting what teachers deem a “fairer” measure of 
success than student proficiency alone. Teachers and schools with weaker students can 
still make “expected growth” regardless of the level perfonnance of students and vice 
versa; teachers with high achieving students do not always produce “expected growth” in 
their students. 

Across the state of North Carolina, 63.6% of students on average achieved a Level III or 
IV score on the biology EOC in 2004-05. In WCPSS, 74.2% of students passed the 
biology EOC. These scores are still considered far from successful, particularly when 
only 46.7% of Black students earned proficient scores. Additional measures are needed 
to identify teachers who are producing the most growth in students. The state does not 
report any measure of growth by student, as the accountability system is based on school 
averages, not individual student scores. 

In the district 

The state provides test analysis software to every district in the state that can be used to 
run school-level results and some simple disaggregations. The state also posts 
disaggregated results on a website. These analyses, however, are limited to average scale 
scores, percentages of students tested who attain proficiency, expected growth, and high 
growth values by subject. Although these statistics provide some useful information to 
teachers and principals, WCPSS evaluators felt that other analyses of student 
achievement data could present a more useful picture of the success of teachers and 
students. If all schools in the district are attaining either high growth or below expected 
growth in a subject area, there is a need to know which schools are producing the most 
growth in order to share the best practices of the district and give a vision of potential 
performance to the lowest-perfonning schools. 

Since school year 1999-2000, this school district has computed residual scores for all 
state test scores of students who have the necessary pretests. These residuals give a 
measure of how students perfonned compared to other similar students (i.e. students with 
the same pretest scores and program identifiers). These residual scores are produced 
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using regression analysis with the current year’s test scores as the dependent variable and 
the state-designated pretest scores as independent variables. Indicator variables control 
for special program status (level of service, e.g. self-contained), free or reduced-price 
lunch status, academic gifted status, and percentage of free or reduced-price lunch 
students in the school. The residual score for a student is the difference between the 
student’s actual score and the predicted model score. 

In each subject, the student residuals are averaged across all students in the school, and a 
standardized z-score (“effectiveness index”) is found for each school by subject. If the z- 
score is greater than 1 , then the school knows that its students have made significantly 
more growth than the other students in the district who have like pretests and program 
codes. Similarly, if the effectiveness index is less than -1, then the students have made 
much less growth than other students like them. Values between -1 and +1 are within 
one standard deviation of the district average. 

Principals receive rosters of student residuals by teacher, course, and section. The 
standard deviation of these residual scores is displayed at the bottom of the roster. 

Student residuals above one standard deviation are coded in green, and student residuals 
below one standard deviation are coded in red. An average residual for the class is 
provided also at the bottom of the roster. 

Table 1 is a sample roster for a 2004-05 biology class of 17 students. For each student, 
the predictor scores are shown. As mentioned above, the predictors for the biology EOC 
are the 8th-grade reading and mathematics scores. The roster then displays the EOC 
scale score and the residual score for each student. These residuals are averaged and an 
average residual score for the class is provided. The average residual of this class was 
3.71 . The principal and teacher can then determine how successful students were on the 
EOC as compared with other students with similar characteristics. Notice that Student 3 
and Student 6 have the same scale score on the biology EOC exam, but Student 3 has a 
negative residual while Student 6 has a positive residual. Student 7 has a lower scale 
score but a higher residual than either Student 3 or 6. The residual shows a measure of 
performance as related to previous performance and other educational indicators, and 
gives a sense of the relative growth for each student. 
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TABLE 1 

Sample High School EOC Residual Roster 


Name 

8th Grade EOG 
Reading Scale Score 

8th Grade EOG 
Math Scale Score 

2005 

BI 

Scale score 

2005 

BI 

Residual 

Student 1 

262 

268 

56 

-0.53 

Student 2 

263 

275 

70 

11.21 

Student 3 

270 

290 

63 

-3.90 

Student 4 

264 

276 

68 

8.46 

Student 5 

269 

279 

68 

6.20 

Student 6 

271 

276 

63 

2.16 

Student 7 

268 

264 

62 

5.36 

Student 8 

256 

274 

51 

-5.31 

Student 9 

278 

278 

66 

1.80 

Student 10 

164 

269 

61 

3.72 

Student 1 1 

No score 

No score 

62 

No residual 

Student 12 

165 

271 

68 

9.64 

Student 13 

259 

270 

62 

4.35 

Student 14 

No score 

No score 

55 

No residual 

Student 15 

270 

277 

66 

4.42 

Student 16 

No score 

No score 

55 

No residual 

Student 17 

159 

255 

54 

4.36 


EOG End of Grade Exam 
BI Biology 


At the Teacher Level 

The student residual scores and the effectiveness indices give the district a comparison 
basis for schools and students. Yet, except for the classroom residual averages, the 
student residuals had not been averaged or standardized at the teacher level. In the past, 
teachers were encouraged to study their rosters for trends in student performance and 
some principals had compared teachers within their school, but no district-wide 
comparisons had been made. This study was a first attempt at identifying the success of 
teachers, as indicated by average residuals, and then to identify the specific aspects of the 
practice of highly effective and relatively less effective teachers in order to isolate 
teachers’ classroom practices that may be associated with high student achievement, 
method. 
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METHOD 


CONTEXT AND PARTICIPANTS 

This research study took place in WCPSS, a large urban/suburban school district in North 
Carolina. The student population is growing rapidly, with an enrollment of 
approximately 120,000 in 2005-06. There were 16 high schools in the study; 14 were 
using a block schedule with two semesters per year (i.e. a biology course is 18 weeks 
long, 90 minutes per day). One school was on an alternating day block schedule (a 
course is 18 weeks in length but taught throughout a full school year.) One other school 
was on a seven-period full-year schedule with 50-minute classes. Fifteen schools offered 
both honors and academic (average) biology. One school taught biology in non-leveled 
heterogeneous classes. Most biology students in these schools were in ninth or tenth 
grade. 

The biology teachers at all 16 high schools participated in the study to some extent. Site 
visits were made to all schools and biology team interviews were held. Forty-three of 
these teachers were the primary focus of the analysis. These teachers had taught at least 
one section of biology in four consecutive school years, beginning in 2001-02. Thus, a 
large amount of student achievement data for each of the teachers was available. These 
teachers had between 4 and 34 years of experience teaching biology. Every high school 
in the district had at least one teacher in the sample, and one high school had five 
teachers. 

INSTRUMENTS, DATA COLLECTION, AND ANALYSIS PROCEDURES 

Biology student residual scores were averaged by teacher for all students taught by the 43 
teachers of interest. The averages were rank-ordered from highest to lowest, yielding a 
roster of “most effective” to “least effective” biology teachers. In addition to these 
student achievement scores, three other types of data were collected. First, a survey was 
prepared and distributed to the 43 study teachers. The teachers answered 35 written 
survey questions concerning preparation, planning, use of time, schedules, use of data, 
and student interaction. 

Second, one of the two principal investigators for this study also visited each of the high 
schools in the district making observations and conducting group interviews with the 
biology teachers. The observation instrument and an interview protocol were adapted 
from the National Science Foundation Coastal Rural Systemic Initiative (CRSI). Both of 
the observers were trained in supervisory observations. 

The 43 teachers who had three consecutive years of student test data in biology from 
2002 to 2004 were randomly assigned an identification number and the 16 high schools 
were also assigned a random identification number. The 10 teachers with the highest 
average residual values were designated as “top” teachers and the 10 teachers with the 
lowest average residual values were designated as “bottom” teachers. Although 43 
teachers were surveyed, the survey analysis focused on these top and bottom teachers. 
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Classroom observations were made of these 20 teachers during the school visits. Results 
of the group interviews were reported, as promised, to both the teachers and school 

administration. These results were considered to be beyond the targeted point of the 
study and a way to shield the identity of the top and bottom teachers. A list of the 
rankings of the teachers was not released. 

RESULTS 


RESIDUALS 

The identification of teachers with highest and lowest average student residuals was a 
stable measure across the three years and into the fourth observation year. After testing 
in May 2005, the top 10 teachers of the previous three years again had high average 
residuals and the bottom 10 teachers had low average residuals when compared to all 
district biology teachers. The teacher with the highest average residual and the teacher 
with the lowest average residual were the same two teachers all four years when 
compared to all teachers teaching biology (not just the teachers who taught it for three 
consecutive years, see Table 2). The top teacher had taught in two schools during the 
span of study years. The top 10 teachers were in seven schools, the bottom 10 teachers 
were in seven schools, and two schools had both top and bottom teachers. 

TABLE 2 

Average Residual Scores for Top and Bottom Teachers 


Teacher 

Residual Average 
2002-04 

Ranking in Study of 43 
teachers 

Residual Average 
2005 

Ranking out of 80 in 
2005 

33 

4.905 

1 

4.09 

1 

17 

4.033 

2 

3.74 

3 

25 

3.437 

3 

1.540 

14 

10 

3.178 

4 

1.882 

9 

34 

2.800 

5 

1.601 

12 

18 

2.557 

6 

2.290 

6 

15 

2.299 

7 

1.507 

16 

43 

2.277 

8 

0.837 

23 

39 

1.862 

9 

0.791 

24 

42 

1.627 

10 

0.549 

28 

29 

-0.819 

34 

-2.008 

68 

40 

-1.008 

35 

N/A* 

N/A* 

36 

-1.474 

36 

-1.186 

60 

21 

-1.565 

37 

-1.622 

66 

2 

-1.845 

38 

-2.916 

78 

30 

-2.058 

39 

N/A* 

N/A* 

24 

-2.309 

40 

-0.785 

55 

31 

-2.433 

41 

-2.894 

77 

4 

-2.784 

42 

-2.332 

74 

20 

-3.051 

43 

-3.221 

80 


*these two teachers did not teach biology in 2004-05 
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As expected, high student scale scores did not always equate to high student residuals. 
There were teachers in the top, bottom, and middle of average student residuals who had 
high or low student performance. For example, a teacher whose students were 99% 
proficient was ranked 20th in effectiveness based on residuals. Disaggregating the 
average residuals by student ethnicity also showed some variance in rankings. One top 
10 teacher was ranked 20th for Black students (see Table 3). 


TABLE 3 

Comparison of Teacher Residual Rankings to Percentage of Students Scoring at 

Proficiency 


Teacher 

Overall 

Residual 

Ranking 

White 

Residual 

Ranking 

Black 

Residual 

Ranking 

Overall % at 
Level III or 
IV 

White % at 
Level III or 
IV 

Black % at 
Level III or 
IV 

17 

2 

2 

1 

94 

97 

84 

18 

6 

5 

20 

87 

92 

57 

39 

9 

10 

8 

87 

95 

64 

42 

10 

9 

10 

75 

91 

60 

12 

20 

20 

18 

99 

100 

95 

2 

38 

38 

36 

64 

87 

49 

31 

39 

43 

37 

81 

90 

40 

21 

41 

30 

41 

71 

90 

48 

20 

43 

42 

43 

54 

86 

28 


COMPARING TOP WITH BOTTOM 

Comparing the survey data from top teachers and bottom teachers was a difficult and 
multifaceted task. There was much variance between teachers on all measures. 
Generalizations were found, yet there was an exception to every generalization. 

Experience and Assigned Instructional Time 

The years of experience teaching biology ranged from 6 to 32 for top teachers and from 4 
to 30 for bottom teachers. The median was 18.5 years for the top teachers and 9.0 years 
for the bottom teachers. Top teachers generally had more experience than bottom 
teachers; yet there was a first-year teacher among the teachers with top residual averages 
in May 2005 and a bottom teacher with 30 years of experience. 


9 




Effective Biology Teaching 


E&R Report No. 05.28 


From 2001 to 2004, the top teachers reported a mean of 1.6 different courses taught 
compared, to a mean of 2.0 courses for bottom teachers. Five top teachers (50%) taught 
only biology compared to 2 bottom teachers (20%). Top teachers averaged 83.4% of 
their instructional time in biology, compared to an average of 64.7% for bottom teachers. 
Top teachers taught 0 to 16 honors sections in three years, with three teachers teaching all 
honors, one teaching all academic, and seven teaching more honors than academic. 
Bottom teachers taught 0 to 7 honors sections, with one teacher teaching only honors, 
three teaching only academic, and one teaching more honors than academic (see Tables 4 
and 5). 


Table 4 

Experience and Instructional Time of Top Teachers 


Teacher 

Overall 

Residual 

Ranking 

# of Years 

Teaching 

Biology 

# of 

Different 

Courses 

Taught 

% of 

Instructional 
Time in 
Biology 

# of Sections 

Honors 

Biology 

# of Sections 

Academic 

Biology 

33 

1 

32 

2 

74 

11 

0 

17 

2 

8 

2 

66 

7 

5 

25 

3 

29 

2 

70 

12 

0 

10 

4 

28 

1 

100 

16 

0 

34 

5 

20 

1 

100 

8 

4 

18 

6 

10 

2 

87 

5 

10 

15 

7 

7 

1 

100 

11 

5 

43 

8 

10 

3 

50 

6 

5 

39 

9 

6 

1 

87 

4 

10 

42 

10 

17 

1 

100 

0 

9 


Table 5 

Experience and Instructional Time of Bottom Teachers 


Teacher 

Overall 

Residual 

Ranking 

# of Years 

Teaching 

Biology 

# of 

Different 

Courses 

Taught 

% of 

Instructional 
Time in 
Biology 

# of Sections 

Honors 

Biology 

# of Sections 

Academic 

Biology 

29 

34 

6 

2 

42 

0 

7 

40 

35 

10 

2 

44 

2 

4 

36 

36 

11 

3 

50 

0 

8 

21 

37 

6 

2 

83 

7 

7 

2 

38 

8 

3 

75 

6 

6 

30 

39 

25 

2 

51 

0 

8 

24 

40 

15 

2 

81 

1 

12 

31 

41 

30 

2 

21 

4 

0 

4 

42 

4 

1 

100 

4 

9 

20 

43 

4 

1 

100 

6 

12 
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Use of Instructional Time 

Analysis of the teacher questionnaires revealed much about classroom practice. Top 
teachers focused most of their class time in lecture and lab. Of the top teachers, 80% 
spent 50% or more class time in these two activities, compared to 40% of bottom 
teachers. There was little difference between top and bottom teachers on using labs (top 
teachers ranged from 10% to 45% of instructional time with a median of 22.5%, 
compared to a range of 10% to 50% and a median of 20% for bottom teachers). There 
was a noticeable difference in the use of lecture (top teachers’ use ranged from 20% to 
60% with a median of 32.5% (see Table 6), while bottom teachers ranged from 10% to 
50% with a median of 20% (see Table 7)). 


Table 6 

Use of Instructional Time of Top Teachers 


Teacher 

Overall 

Residual 

Ranking 

Lecture 

Labs 

Whole 

Group 

Discussion 

Demos 

Projects 

Small 

Group 

Discussion 

Partner 

Work 

Testing 

Other 

33 

1 

50% 

15% 

0% 

5% 

0% 

15% 

0% 

8% 

7% 

17 

2 

30% 

25% 

3% 

7% 

3% 

6% 

15% 

11% 

0% 

25 

3 

60% 

20% 

5% 

5% 

0% 

0% 

0% 

10% 

0% 

10 

4 

25% 

35% 

5% 

1% 

3% 

15% 

10% 

7% 

0% 

34 

5 

35% 

15% 

10% 

2% 

1% 

17% 

18% 

2% 

0% 

18 

6 

20% 

20% 

5% 

0% 

0% 

10% 

20% 

15% 

0% 

15 

7 

20% 

20% 

5% 

5% 

5% 

5% 

15% 

15% 

10% 

43 

8 

20% 

45% 

10% 

5% 

5% 

5% 

5% 

5% 

0% 

39 

9 

60% 

10% 

0% 

5% 

5% 

0% 

10% 

10% 

0% 

42 

10 

50% 

10% 

0% 

0% 

0% 

10% 

10% 

10% 

10% 


Table 7 

Use of Instructional Time of Bottom Teachers 


Teacher 

Overall 

Residual 

Ranking 

Lecture 

Labs 

Whole 

Group 

Discussion 

Demos 

Projects 

Small 

Group 

Discussion 

Partner 

Work 

Testing 

Other 

29 

34 

20% 

25% 

5% 

5% 

5% 

10% 

20% 

10% 

0% 

40 

35 

30% 

20% 

0% 

0% 

0% 

20% 

20% 

10% 

0% 

36 

36 

10% 

10% 

5% 

5% 

30% 

15% 

15% 

10% 

0% 

21 

37 

10% 

50% 

0% 

1% 

5% 

5% 

9% 

10% 

10% 

2 

38 

10% 

10% 

20% 

5% 

30% 

10% 

10% 

5% 

0% 

30 

39 

10% 

20% 

10% 

0% 

5% 

30% 

10% 

5% 

10% 

24 

40 

15% 

25% 

9% 

10% 

5% 

30% 

10% 

5% 

0% 

31 

41 

40% 

25% 

5% 

2% 

10% 

3% 

5% 

10% 

0% 

4 

42 

50% 

10% 

10% 

5% 

5% 

10% 

5% 

5% 

0% 

20 

43 

20% 

20% 

5% 

5% 

10% 

20% 

10% 

10% 

0% 
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There was also an apparent difference in the use of class time for projects, small group 
discussion, and partner work. The top teachers’ use ranged from 0% to 36% with a 
median of 22%. The bottom teachers’ reported time ranged from 18% to 60% with a 
median of 40%. The top teachers also reported more testing. Their percentage of time 
devoted to testing ranged from 2% to 15% with a median of 10%, compared to a range of 
5% to 10% and a median of 7.5% for bottom teachers (four bottom teachers reported 5% 
and six reported 10%) (see Tables 6 and 7). 

While the term “lecture” may cover a variety of activities, it seems clear that the top 
teachers exercised more control over time use and student attention. These teachers 
tended to spend more time on teacher-controlled activities but relatively less class time 
on projects, partner work, and small-group discussion. 

Planning 

Seven of the top teachers said that they planned for instruction with one or more 
teachers, but six bottom teachers reported always planning alone. The district provides a 
pacing guide for biology instruction with suggested classroom tasks. Of the top teachers, 
only one teacher (who had participated in writing the pacing guide) reported using most 
of the suggested tasks. Nine top teachers used some or none. Of the bottom teachers, 
five marked most or many, four some, and only one none. Top teachers reported 
planning most of their activities themselves, while more bottom teachers used resources 
already prepared. 

Use of Data 

All 10 top teachers reported that they used data in the planning of instruction, while three 
bottom teachers stated they had no time for data (including one department chair of a 
school with a low effectiveness index). Nine of the 10 top teachers reported using data 
that they collected, and nine used data from the school and district administration. Two 
top teachers were at a school that had developed common classroom assessments that 
were used to report proficiency on biology goals to students and to provide goal-based 
regular structured remediation to students. 

The school with the highest district effectiveness index (two of the top teachers taught in 
this school) used goal summary reports provided by the state testing program from 
previous years to plan the pacing and delivery of instruction. A “year at a glance” 
document was written. The instruction was goal-driven, not book-driven. This was also 
the school where a first-year teacher had a top ranked residual average in 2005. 
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Barriers to Effectiveness 

Top teachers were generally of one mind when asked about the main barrier to their 
effectiveness with students. Nine gave answers related to time. Six cited lack of class 
time with students (too much material in the curriculum) and two cited lack of time to 
plan and meet with students (too many other duties). One top teacher said that there was 
too much in the curriculum (causing a time issue). One teacher thought students did not 
study. This answer was the only response not related directly to time with students. 

The bottom teachers did not show a similar consensus of answers. There were seven 
different answers. Three cited ‘Time” with no explanation, two said “too many other 
tasks, too much paper work, and surveys”, two said “too much in curriculum”, one cited 
parental home structure, one said “students can’t read, student motivation, and class 
size”, one said “lack of colleague support”, and one gave no answer. 

Classroom Observations 

WCPSS’ science curriculum specialist, one of the two researchers conducting this study, 
conducted observations in the classes of most and least effective teachers. His overall 
observations of most effective teachers were that they: 

• participated in group planning. 

• worked on and used a common pacing guide. 

• held EOC review sessions by selected content. 

• made data driven decisions about which goals and objectives to stress. 

• conducted frequent assessments. 

• communicated to students their progress on state standards. 

• were in schools that had strong departmental leadership. 

• designed a “year at a glance” document. 

• presented themselves as professionals (they were well dressed). 

CONCLUSIONS 

The main goal of this research study was to develop a value-added instructional 
improvement analysis model that was a collaborative effort of the WCPSS Evaluation 
and Research Department and the Curriculum and Instruction Department. This goal was 
accomplished. The year-to-year consistency of teacher residual rankings in the district 
showed that teachers who produce the most growth in students on EOC tests can be 
identified. It should be remembered that this is a high performing district and that even 
teachers on the lower end of the residual rankings may be more effective than other 
teachers in the state, but with regard to local standards of perfonnance, they are 
producing less growth in their biology students. The surveys and observations of these 
teachers provided information about some of the classroom practices that may be 
contributing to growth. The results of this study were shared with principals and 
teachers. Understanding of and confidence in residual analysis has been heightened 
throughout the district. Collaboration between the two departments involved continues, 
and an Algebra I project is in progress. 
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Since the residual averages were shown to be stable, the average residuals for all EOC 
and End-of-Grade (EOG) subjects were analyzed in order to provide principals and 
teachers with the values above which, and below which 25% of all teacher residual 
averages fall. For example, in Biology, if a teacher’s residual average is above 1, the 
teacher’s residual average is among the top 25% of teacher averages in the district, and if 
below -1, among the bottom 25% of teacher averages. A handout explaining residual 
analysis and the school effectiveness index has been prepared and widely distributed. 

The survey and observation results were shared across the district and the state. Key 
conclusions about the most effective teachers of this study were that they: 

• focused on the delivery of biology instruction. 

• resisted distractions from their classroom efforts. 

• maximized student use of class time. 

• studied and planned with other teachers using the North Carolina 
Biology Standard Course Of Study. 

• focused all student time on the Standard Course of Study goals. 

• planned students’ activities that are always under teacher control. 

• used data to guide their instructional practice. 

Key conclusions about the least effective teachers of this study were that they: 

• had less focus on Biology. 

• spent less time on planning. 

• planned alone and used prepared lessons from the district pacing 
guide. 

• took less time to prepare their own lessons. 

• used a wider variety of activities in the classroom and had students 
spend more time in student led activities. 

• were not always focused on the Standard Course of Study. 

DISCUSSION 

The findings of this study concerning teacher directed use of time in alignment with a 
curricular focus has been supported by many earlier research studies (Anderson & 
Walberg, 1994; Frederick, 1980; Frederick & Walberg, 1980; Walberg, 1999; Walberg & 
Frederick, 1991). The predominant use of direct teaching as an effective tool in 
promoting student learning is also well documented in the research (Gage & Needles, 
1989; Walberg, 1999; Wang, Haertel, & Walberg, 1993a, 1993b). 

Berlin’s (1993) essay, The Hedgehog and the Fox, as explained by Collins (2001) can 
also be used to describe the top teachers as focused “hedgehogs” with a single- 
mindedness of purpose contrasted with the bottom teachers as foxes, whom Collins 
(2001) writes, “ pursue many ends at the same time” (page 91). Hedgehogs are more 
likely to produce quality results because of their focus on the goal (one goal). 
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Yet these are general conclusions, and there was a top teacher and a bottom teacher in the 
study who were exceptions to each of these conclusions. The largest impact of this study 
may be found in the discussions around classroom practice that have begun. Teachers 
have either been validated in their practice or have been shown that there are teachers 
who are producing more growth in students. Middle and bottom teachers can no longer 
say that their students are doing the best that they can; now these teachers know that there 
are similar students performing better in other classes. These teachers have been 
provided with a vision of better perfonnance, thus creating possibilities for improvement. 

Two schools in the district with low biology effectiveness indices and teachers with low 
middle to bottom average residuals has begun biology improvement cycles. They have 
contacted three schools with higher effectiveness indices and are using the information 
gained from these schools and their own residual averages to drive their improvement. 
This effort models for our district the beginnings of practical use of this new value-added 
instructional improvement analysis model piloted by this study. 

IMPLICATIONS 

This research made a connection between test scores and classroom practice that had 
been lacking. Effectiveness indices and average student residuals are now beginning to 
be seen as indicators of where to look for best practices and models for improvement 
beyond a classroom and a school. Conversations have begun that break through the 
isolation that teachers and administrators sometimes feel. 

The challenge now is to find ways to help teachers find time to study their data, plan 
together, and to engage in these discussions. There is also the need to move more of this 
information to the classroom, as not all principals and schools are using the residual data. 
Since the most effective teachers resist distractions from class time, the curriculum 
specialists need to look for unintrusive ways to share the practice of these teachers. 
School- wide improvement projects need to documented and later shared. 

The survey and observation procedures can be improved in upcoming studies. Some of 
the survey questions did not provide useful insights, and there were questions that arose 
but could not be answered by the instruments of this study. A recently initiated Algebra I 
study will look more deeply into classroom practices. A survey and classroom 
observations will assist to study the use of research-based strategies (Marzano, Pickering, 
and Pollock, 2001). Class size and number of student contacts will also be studied. 

This study is the first step in a district-wide school improvement effort that will seek to 
identify best teaching practices. Sharing across schools will become the nonn, not the 
exception. A good district can become a great district. As Collins (2001) writes, 
“Greatness, it turns out, is largely a matter of conscious choice” (p. 1 1). 
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